Color shading correction device, color shading correction method, and color shading correction program

ABSTRACT

An image processing device that performs color shading correction on an image formed by a plurality of pixel signals with correction data includes: a correction execution decision unit that decides whether or not color shading correction is to be executed for each pixel signal in order to perform color shading correction for a part of the plurality of pixel signals and not to perform color shading correction for another part of the plurality of pixel signals; and a correction unit that performs correction of the pixel signal of which execution of correction is decided by the correction execution decision unit for each color component thereof.

This application is a continuation of International Application No.PCT/JP 2007/051781 filed Feb. 2, 2007

INCORPORATION BY REFERENCE

The disclosures of the following applications are herein incorporated byreference:

-   Japanese Patent Application No. 2006-026679 filed Feb. 3, 2006-   International Application No. PCT/JP 2007/051781 filed Feb. 2, 2007

BACKGROUND OF INVENTION

1. Field of the Invention

The present invention relates to an image processing device, an imageprocessing method, and an image processing program that perform colorshading correction of photographic images.

2. Description of Related Art

A camera is known which corrects color shading, i.e., becoming reddishor bluish in a peripheral part as compared with a central part of animage, depending on an aperture value, an exit pupil distance, and adistance from a center of an imaging area of a photographic lens (see,for example, Patent Reference 1). This camera retrievesaperture-dependent characteristics information that depends on theaperture value upon imaging and shading correction information thatdepends on exit pupil distance and exit pupil diameter upon imaging froma table for a microlens and calculates R, G, and B gain control amountsbased on the focal length and lens characteristics information(information indicating characteristics of peripheral light reduction).

-   Patent Reference 1: Japanese Laid-Open Patent Application No.    2000-324505.

SUMMARY OF THE INVENTION

However, there arises a problem that calculation of correction data foreach of R, G, and B colors on the whole screen depending on photographicconditions and distance from the center of the screen (image height) andperformance of correction based on the calculated data result in anincrease in the amount of calculation, so that load of the correctionprocessing is increased.

In addition, an aperture-dependent characteristics information table anda shading characteristics information table in accordance with thecharacteristics of microlenses preliminarily mounted in each camera arekept in the camera. As a result, a large capacity memory is necessaryfor recording a large number of tables for preparing correction data.

On the other hand, when the cameras of the same model are used,sufficient correction cannot be performed if color shading occursdifferent in different camera due to individual variability.

According to a first aspect of the present invention, an imageprocessing device that performs color shading correction on an imageformed by a plurality of pixel signals with correction data, the deviceincludes: a correction execution decision unit that decides whether ornot color shading correction is to be executed for each pixel signal inorder to perform color shading correction for a part of the plurality ofpixel signals and not to perform color shading correction for anotherpart of the plurality of pixel signals; and a correction unit thatperforms correction of the pixel signal of which execution of correctionis decided by the correction execution decision unit for each colorcomponent thereof.

According to a second aspect of the present invention, it is preferredthat in the image processing device of the first embodiment, the pixelsignal for which color shading correction is to be executed formsrespectively peripheral area of the image, and the pixel signal forwhich color shading correction is not to be executed forms respectivelycentral area of the image.

According to a third aspect of the present invention, it is preferredthat the image processing device of the first embodiment furtherincludes: a correction data decision unit that decides the correctiondata for each color component of the pixel signal using an opticalphotographic condition when a correction target image is obtained, andin this device the correction execution decision unit decides whether ornot correction is to be executed for each pixel signal that forms thecorrection target image depending on the optical photographic condition;and the correction unit performs correction of the pixel signal of whichexecution of correction is decided by the correction execution decisionunit for each color component thereof based on the correction data.

According to a fourth aspect of the present invention, it is preferredthat in the image processing device of the third embodiment the opticalphotographic condition includes an exit pupil position and an aperturevalue when the correction target image is obtained, the correctionexecution decision unit decides whether or not correction is to beexecuted for each of the pixel signals that form the correction targetimage depending on the exit pupil position and the aperture value, andthe correction unit performs correction of the pixel signal of whichexecution of correction is decided by the correction execution decisionunit for each color component based on the correction data.

According to a fifth aspect of the present invention, it is preferredthat in the image processing device of the forth embodiment, thecorrection execution decision unit decides that correction is to beexecuted when each pixel that constitutes the image sensor has an imageheight larger than a predetermined value determined depending on theexit pupil position and the aperture value.

According to a sixth aspect of the present invention, an imageprocessing device that performs color shading correction on an imageformed by a plurality of pixel signals with correction data, the deviceincludes: a correction data obtaining unit that obtains the correctiondata suitable for an optical photographic condition for each of theplurality of pixel signals using a first image obtained by photographyof a subject having a color distribution that is known in advance; and acorrection unit that corrects a plurality of pixel signals that form asecond image for each color component thereof based on an opticalphotographic condition when the second image is obtained and thecorrection data.

According to a seventh aspect of the present invention, it is preferredthat in the image processing device of the sixth embodiments, thecorrection data obtaining unit obtains the correction data for eachcolor component suitable for a predetermined optical photographiccondition for each of the plurality of pixel signals using the firstimage obtained by photography of the subject having a known colordistribution under the predetermined optical photographic condition, andthe correction unit corrects for each color component a plurality ofimage signals that form the second image obtained under the same opticalphotographic condition as the predetermined optical photographiccondition based on the correction data.

According to an eighth aspects of the present invention, it is preferredthat the image processing device of the sixth embodiments furtherincludes an updating unit that updates data relating to correctionstored in advance based on the first image to generate the correctiondata, and in this device the correction unit corrects for each colorcomponent a plurality of pixel signals that form the second image basedon the optical photographic condition when the second image is obtainedand the updated correction data.

According to a ninth aspect of the present invention, it is preferredthat in the image processing device of the sixth to the eighthembodiments, the subject having a known color distribution is a charthaving a uniform color distribution.

According to a tenth to an eighteenth aspect of the present invention,an image processing method performs processing that corresponds to theimage processing device according to any one of the first to the ninthembodiments.

According to a nineteenth aspect of the present invention, an imageprocessing program causes a computer to execute an image processingmethod according to any one of the tenth to the eighteenth aspects.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an external view of an electronic camera according to anembodiment of the present invention;

FIG. 2 is a block diagram illustrating the electronic camera accordingto the embodiment of the present invention;

FIG. 3 illustrates the configuration in the vicinity of an image sensorin the electronic camera according to the present invention;

FIG. 4 illustrates the spectral sensitivity of the image sensor;

FIG. 5 illustrates an example in which the transmittance of an infraredcut filter varies depending on the incidence angle;

FIG. 6 illustrates the variation of spectral sensitivity of an imagesensor depending on the incidence angle due to the characteristics ofthe infrared cut filter and of the color filter;

FIG. 7 illustrates dependence of the incidence angle to the image sensoron the exit pupil distance and the distance from the image sensor aswell as the image height;

FIG. 8 presents a calculation table for calculating an exit pupilposition for each lens model, which is a data for correction necessaryfor obtaining color shading;

FIGS. 9A to 9D present calculation tables for calculating, FIG. 9Aillustrating coefficient a of an R gain function, FIG. 9B illustratingcoefficient b of the R gain function, FIG. 9C illustrating coefficient cof the B gain function, and FIG. 9D illustrating coefficient d of the Bgain function, which are each a data for correction necessary forobtaining color shading;

FIG. 10 is a flowchart illustrating color shading correction processingin the first embodiment of the present invention;

FIG. 11 is a block diagram illustrating the configuration of a main partof a personal computer in a second embodiment;

FIG. 12 is a flowchart illustrating color shading correction processingin the second embodiment;

FIG. 13 is a block diagram illustrating the configuration of a main partof the electronic camera according to a modification of the secondembodiment;

FIG. 14 is a flowchart illustrating color shading correction processingaccording to a modification in the second embodiment;

FIG. 15 is a flowchart illustrating calculation processing forcalculating a correction function in a third embodiment;

FIG. 16 is a flowchart illustrating the color shading correctionprocessing in the third embodiment; and

FIG. 17 is a diagram illustrating a configuration of equipment in wholeused for providing a program product.

DESCRIPTION OF PREFERRED EMBODIMENTS First Embodiment

Referring to FIGS. 1 to 10, a camera having mounted thereon the imageprocessing device according to a first embodiment of the presentinvention is described. As shown in FIG. 1, an electronic camera 1includes a camera body 10, on an upper surface of which are provided amain switch (power switch) 2 that turns ON/OFF power supply to theelectronic camera 1, a release button 3 that instructs photographing, amode dial 4 for setting photographic conditions and action modes such asa photographic mode and a reproduction mode, a built-in flash unit 5that illuminates a subject, and an accessory shoe for attaching adetachable external flash device to the camera body 10.

On a front face of the camera body 10, there are provided a mount 7 formounting thereon an interchangeable lens, a dismounting button 8 fordismounting the interchangeable lens mounted on the mount 7, and apreview button 9 for performing a preview action by which an aperture ofthe interchangeable lens is reduced to a set aperture or a controlledaperture before image capturing. On a rear face of the camera body 10,there are provided a finder (not shown) through which a subject imagebefore image capturing is confirmed and a liquid crystal monitor (notshown) that displays the captured image and various settings.

FIG. 2 is a block diagram illustrating a control system of theelectronic camera 1. A calculation circuit 101 includes a micro computerand so on. The calculation circuit 101 performs predeterminedcalculations based on signals output from various circuits andcomponents detailed hereinbelow and input to the calculation circuit andoutputs control signals based on the results of the calculations to thevarious circuits and components.

An image sensor 121 includes, for example, a CCD image sensor or a CMOSimage sensor. The image sensor 121 captures an image formed by lightfrom a subject that passes through an interchangeable lens L forphotographing and outputs an image signal to an A/D conversion circuit122. The A/D conversion circuit 122 converts analog image signals todigital signals. The image sensor 121 and the A/D conversion circuit 122are each driven at predetermined action timing by a drive signal outputfrom the timing circuit 124. A lens information transmission circuit 127outputs to the calculation circuit 101 lens information including a lenstype, a focal length, a lens position (camera-to-subject distance) andso on that have been input through the mount 7.

An image processing circuit 123 includes ASIC and so on. The imageprocessing circuit 123 performs, besides image processing such as whitebalance processing to image data after conversion into digital data,compression processing in which the image data after the imageprocessing is compressed, and expansion processing in which thecompressed image data are expanded. A buffer memory 125 temporarilystores image data to be processed by the image processing circuit 123. Arecording medium 126 records the image data after the image processing.A liquid crystal monitor 128 displays an image corresponding to theimage data stored in the buffer memory 125 according to various settingsand the above-mentioned action and in a reproduction mode, displays animage corresponding to the image data recorded in the recording medium126.

FIG. 3 schematically shows the configuration of a periphery of the imagesensor 121 and a variation of inclination of incident light to the imagesensor 121. FIG. 3 shows three photodiodes 130 a to 130 c (hereinafter,130 is assigned when generically named) of the image sensor 121 and R,G, and B color filters 134 a to 134 c (hereinafter, 134 is assigned whengenerically named). The color filters 134 a to 134 c include microlenses131 a to 131 c (hereinafter, 131 is assigned when generically named),respectively, on front faces thereof in order to increase lightreceiving sensitivity. Each pixel of the image sensor 121 includes aphotodiode 130, a color filter 134, and a microlens 131 and so on. Themicrolenses 131 include on their front face, i.e., on the side of asubject various filters such as an infrared ray cut filter 132 a and anultraviolet ray cut filter 132 b. The filters 132 a and the 132 b aregenerically named a filter 132. The filter 132 includes a glass platecoated with a thin film coat. Note that 133 designates an exit pupil ofthe interchangeable lens attached to the camera.

As can be seen from FIG. 3, the inclination of main light 136 b of light135 b that is incident to a pixel in the vicinity of the optical axis isapproximately parallel to the optical axis. The inclination of the mainlight 136 a of the light 135 a that is incident to a pixel in theperipheral part of the screen is a predetermined angle. This inclinationis larger for the pixel closer to the peripheral part of the screen.

When photographic light enters each photodiode in the peripheral part ofthe screen and if the incident angle of the light to the microlens 131and the filter 132 disposed in the front of each photodiode increases,for example, to such an extent that the inclination of the incidentlight relative to the optical axis becomes outside a range of angle inwhich the microlens 131 can collect light, the quantity of lightincident to the pixel decreases. The microlens 131 is constituted by amaterial having a distributed refractive index, so that in a precisesense, the range of angle in which the microlens 131 can collect lightdiffers depending on the wavelength of the light. Therefore, lossamounts of the quantities of R, G, and B lights differ from each other,and this causes color shading to occur. The color shading has anincreasing influence at an increasing aperture value (F number). Inaddition, the color shading provides different influences since theposition of exit pupil 133 is different depending on lens model andcamera-to-subject distance and the inclination of the incident light ischanged. That is, the color shading has a different influence dependingon optical photographic conditions. Also, the color shading hasdifferent characteristics depending on the model of the camera since italso depends on the characteristics of the microlens 131.

On the other hand, the filter 132 has a spectral transmittance thatdepends on the incident angle of the light. This also yields colorshading. This will be explained referring to FIGS. 4 to 6.

FIG. 4 shows the spectral sensitivity of the image sensor 121 thatincludes the photodiode 130, the color filter 134, and the microlens131. FIG. 5 shows how the spectral transmittance of the infrared ray cutfilter 132 a varies depending on incidence angle (θ₀, θ₁). The spectralsensitivity of the image sensor 121 that includes the photodiode 130,the color filter 134, and the microlens 131 and the spectral sensitivityof the image sensor 121 with the infrared ray cut filter 132 a are eachdetermined by a product of the graph shown in FIG. 4 by the graph shownin FIG. 5, respectively.

FIG. 6 shows the spectral sensitivity on this occasion for eachincidence angle. The values of integral of the graphs shown in FIG. 6correspond to light quantities of a red component (hereinafter referredto as R), a green component (hereinafter referred to as G), and a bluecomponent (hereinafter referred to as B) of each pixel. This indicatesthat only the light quantity of R varies depending on the incidenceangle of photographic light. Note that for the sake of simplicity,influences by the ultraviolet ray cut filter 132 b and other constituentelements are neglected in FIG. 6. In the case of the image sensor 121provided with the ultraviolet ray cut filter 132 b, the spectralsensitivity on the shorter wavelength side is also varied depending onthe incidence angle to give influences similar to that in the case ofthe infrared ray cut filter 132 a to the shorter wavelength side.

As mentioned above, the quantity of the light incident to the photodiode130 depends on the incidence angle of the photographic light incident toeach microlens 131 and each filter 132. The incidence angle of the lightincident to each microlens 131 or each filter 132 is greater at agreater image height as shown in FIG. 7 assuming that the exit pupilposition and the aperture diameter are not changed. The incidence angleθ varies depending on optical photographic conditions such as exit pupilposition, aperture value, and so on. Therefore, the incidence angle θcan be defined in terms of three variables, i.e., exit pupil position,aperture value, and image height. In the present embodiment, theinterchangeable lens calculates an exit pupil position referring toTable 1 shown in FIG. 8 stored in an ROM inside the interchangeablelens. That is, the interchangeable lens calculates the exit pupilposition based on the focal length and the camera-to-subject distance,which are optical photographic conditions, referring to Table 1.

Based on the exit pupil position calculated in this manner and theaperture value upon photography, the calculation circuit 101 calculatescoefficient “a” of R-gain(r) referring to Table 2 shown in FIG. 9A andcoefficient “b” of R-gain(r) referring to Table 3 shown in FIG. 9B. Thecalculation circuit 101 calculates coefficient “c” of B-gain(r)referring to Table 4 shown in FIG. 9C and coefficient “d” of B-gain(r)referring to Table 5 shown in FIG. 9D. Tables 2 to 5 are calculationtables for calculating coefficients corresponding to camera models usedin photographing.

By using the coefficients “a” to “d” calculated in this manner, anR-gain-function(r) and a B-gain-function(r) represented by the followingformulae (1) and (2), respectively, are calculated.R-gain-function(r)=ar+b  (1)B-gain-function(r)=cr+d  (2)

In the above formulae, r represents image height of each pixel (distancefrom an optical axis of the interchangeable lens attached to the camerato each pixel).

Based on the formulae (1) and (2) above, the R-gain(r) and the B-gain(r)are calculated for each photodiode, in other words, for each pixel usingits image height r, and an output value (pixel value) Sout of eachphotodiode is corrected for each color component according to thefollowing formulae (3) and (4), respectively.R pixel value after correction=(R pixel value before correctionSout)×(R-gain(r))  (3)B pixel value after correction=(B pixel value before correctionSout)×(B-gain(r))  (4)

In the present embodiment, color shading correction is performed and itis sufficient that a relative amount based on G signal as a standard iscorrected. Therefore, G signals are not corrected based on incidenceangle. However, G signals may be subjected to luminance shadingcorrection in the same manner as conventionally. When luminance shadingcorrection is additionally performed, all the R, G, and B signals aremultiplied by respective correction functions. Assuming that theluminance shading correction function is f(r), the formulae (3) and (4)are converted into formulae (5) to (7).R pixel value after correction=(R pixel value before correctionSout)×(R-gain(r))×f(r)  (5)G pixel value after correction=(G pixel value before correctionSout)×f(r)  (6)B pixel value after correction=(B pixel value before correctionSout)×(B-gain(r))×f(r)  (7)

The focal length, camera-to-subject distance, and aperture value, whichare optical photographic conditions upon photography, are transmitted aslens information from the lens to the camera body. Determination of alens is performed by transmission of a unique identification numberassigned to each interchangeable lens from the lens to the camera body.Table 1 is recorded in the ROM on the lens side. Upon photography, anexit pupil position depending on optical photographic conditions (focallength and camera-to-subject distance) is calculated on the lens sideand the calculated exit pupil position is transmitted to the camerabody. In a predetermined recording region of the calculation circuit101, there is preliminarily stored Table 2 to Table 5 depending on themodel of the electronic camera 1. The calculation circuit 101 calculatescoefficients a to d based on the aperture values and exit pupilpositions transmitted from the lens with reference to Table 2 to table5. When the lens has no calculation function, Table 1 may be prepared ina number corresponding to the models of the lens that can be attached onthe camera body side and the focal length and exit pupil position may becalculated on the camera body side based on the focal length and thecamera-to-subject distance set upon photography.

Hereinafter, a preparation method for preparing Tables 2 to 5 above isdescribed. Using a plurality of lenses having different exit pupilpositions, a subject such as a grey chart having a uniform colordistribution is photographed under uniform illumination while changingthe position of the exit pupil 133 and the aperture value. For imagesobtained by photographing the subject under respective conditions,ratios of R, G, and B, i.e., R/G and B/G are calculated for each pixelvalue as follows.

That is, for each pixel value, ratios of R/G(x₀,y₀) and B/G(x₀, y₀) atthe image center (x₀,y₀) on the optical axis to R/G(x,y) and B/G(x,y),respectively, to prepare R-gain(x,y) and B-gain(x,y), respectively,which are expressed by formulae (8) and (9), respectively, as follows.R-gain(x,y)={R/G(x ₀ ,y ₀)}/{R/G(x,y)}  (8)B-gain(x,y)={B/G(x ₀ ,y ₀)}/{B/G(x,y)}  (9)

In the present embodiment, three image data corresponding to R, G, and Bcolors, respectively, are used. In the case of the camera with colorfilters in Bayer arrangement, image data after correction are used.Therefore, R/G(x₀,y₀) represents R pixel value/G pixel value for onepixel located at the image center (x₀,y₀), and B/G(x₀,y₀) represents Bpixel value/G pixel value for one pixel located at the image center(x₀,y₀). Similarly, R/G(x,y) represents R pixel value/G pixel value fora pixel located at any position (x,y), and B/G (x,y) represents B pixelvalue/G pixel value for a pixel located at any position (x,y). When thenumber of pixels in the vertical or horizontal direction is even, whichone of the pixels is located at the image center (x₀,y₀) can not beuniquely determined. In such a case, the pixel value at the image center(x₀,y₀) may be obtained by interpolation using pixel values of aplurality of pixels in the vicinity of the image center.

As mentioned above, considering that the color shading may be deemed toconsist mainly of rotationally symmetric components, R-gain(x,y) andB-gain(x,y) can be approximated by functions of image height r. In thiscase, R-gain(x,y) and B-gain(x,y) need not be analyzed on the image inwhole and the range of analysis may be limited only to a first quadranttaking into consideration the rotational symmetry of color shading. Theimage height r can be expressed by formula (10) below.r=((x−x ₀)²+(y−y ₀)^(1/2)  (10)

From the formulae (8), (9), and (10), the R gain and the B gain areapproximated by linear functions of the image height r, such as theformulae (1) and (2), respectively. In this case, the coefficients “a”,“b”, and “c” are calculated by fitting or the like from results of imageanalysis (R-gain (x,y) and B-gain(x,y)) of respective opticalphotographic conditions (exit pupil position, aperture value).

In the case of a pixel in the vicinity of optical axis, the influence ofthe color shading can be ignored since the incident angle of light tothe microlens 131 and the infrared ray cut/ultraviolet ray cut filter132 is small. Assuming the maximum incidence angle at which theinfluence of color shading can be ignored, that is, the minimumincidence angle at which correction is necessary, is θmin, then gainscan be calculated as indicated below using a corresponding image heightrmin.

When r>rmin,R-gain(r)=ar+bB-gain(r)=cr+d; andwhen r≦rmin,R-gain(r)=B-gain(r)=1  (11).

Note that the threshold rmin can be calculated from the minimumincidence angle θmin at which color shading correction is necessary, theposition of the exit pupil 133 (distance from an image formation plane)P₀, and aperture value F upon photography according to the followingformula.rmin=P ₀×[tan θmin−tan {sin⁻¹(½F)}]  (12)

Therefore, according to the formula (II) above, processing to reflectthe results of calculations of the R-gain(r) and the B-gain (r) on imageoutput becomes unnecessary, so that the correction of actual images canbe performed efficiently.

The data in Tables 2 to 5 prepared in the above mentioned manner arestored as data for correction in the calculation circuit 101 of thecamera. When color shading correction is performed, respective data arereferenced to obtain the R gain and the B gain. When there is no datafor correction that correspond to actual optical photographic conditionsone for one, data having close conditions are used to obtain the R gainand the B gain by interpolation.

Note that since the influence of the color shading largely depends onthe characteristic of the image sensor 121 and the characteristics ofthe filter 132 as mentioned above, the data for correction such as thoseshown in Tables 2 to 5 are prepared for every image sensor 121 or forevery camera model. When the color shading correction is performed bythe camera body upon photography by the camera, the camera stores a setof correction data corresponding to each camera and performs acorrection using them. On the other hand, when the photographic image isread in by an external calculation device such as a personal computer toperform color shading correction processing therein, correction isperformed by using data for correction that depend on the camera modelused in the photography. Therefore, it is indispensable that thephotographic image data are stored together with appendix informationsuch as exit pupil position (or focal length, camera-to-subjectdistance, or lens type), aperture value, and camera type. Further, whenit is possible to perform color shading correction processing both bythe camera body and by the external calculation device, information thatdiscriminates whether or not color shading correction processing hasbeen completed is attached to the image data.

The color shading correction action of the electronic camera having theabove-mentioned configuration is described referring to the flowchartshown in FIG. 10. Each procedure in FIG. 10 is performed by executing aprogram in the calculation circuit 101. In the present embodiment, theoptical photographic conditions are set as follows.

(1) Name of camera model: A;

(2) Type of camera lens: Lens type I (focal length (zoom) range of f₀-f₁mm, Full F value F2.8);

(3) Focal length upon photography: 50 [m];

(4) Camera-to-subject distance: 4 m; and

(5) Aperture value (F value): F4.

In a step S1, an image of which color shading correction is to beperformed (correction target image) is read in, and the process isadvanced to a step S2. In the step S2, optical photographic conditions,that is, the position of the exit pupil 133, type of lens, focal length,and camera-to-subject distance, transmitted from the interchangeablelens are read in, and the process is advanced to a step S3.

The position of the exit pupil 133 of the lens type I is calculated onthe interchangeable lens side by interpolating the position P₀ of theexit pupil 133 at a focal length of 50 mm and a camera-to-subjectdistance of 4 m using data (P22, P23, P32, and P33) for conditions closeto the above-mentioned optical photographic conditions (3) and (4).

Note that the image sensor 121 is unique to the camera and hence is notinterchangeable. Therefore, on this occasion, Tables 2 to 5 to be usedare set depending on the numerical aperture of the microlens 131 and theincidence angle characteristics of the filter 132 that are provided inthe camera.

In the step S3, a decision is made as to whether or not the receivedposition P₀ of the exit pupil 133 is equal to or larger than apredetermined threshold Pmax. If the result of the decision in the stepS3 is YES, that is, if the position P₀ of the exit pupil 133 is decidedto be equal to or larger than the predetermined threshold Pmax, theprocess is advanced to a step S4 where no color shading correction isperformed, and the process is advanced to a step S12. This is becausewhen the position P₀ of the exit pupil 133 is remoter from the imageformation plane, the incidence angle is smaller, so that the influenceof the color shading can be ignored as is apparent from FIG. 7. If theresult of the decision in the step S3 is NO, that is, if the position P₀of the exit pupil 133 is decided to be smaller than the predeterminedthreshold Pmax, the process is advanced to a step S5.

In the step S5, the aperture value upon photography is read in, and theprocess is advanced to a step S6. In the step S6, a minimum image heightat which correction is necessary (rmin) is determined based on theoptical photographic conditions, and the process is advanced to a stepS7. The image height rmin is calculated according to the above-mentionedformula (12) using the exit pupil position P₀ upon photography of thecorrection target image that has been read in during the step S2 and theaperture value read in during the step S5. In the step S7, color shadingcorrection functions at an image height of r, that is, anR-gain-function (r) and a B-gain-function(r) are calculated from theaperture value read in during the step S5 and the position P₀ of theexit pupil 133 read in during the step S2 according to the formulae (1)and (2), respectively, after each coefficient is determined referring toTables 2 to 5.

A more specific example will be explained in which the position P₀ ofthe exit pupil 133 read in during the step S2 is set P₁<P₀<P₂. First,the above-mentioned coefficients “a”, “b”, “c”, and “d” are obtainedreferring to Tables 2 to 5. Since the optical photographic conditionsare such that the position of the exit pupil 133 is P₀ and the aperturevalue=4, a coefficient a₀ is calculated referring to Table 2 byinterpolating the coefficient a₀ from a coefficient a₂₁ for the positionP₁ of the exit pupil 133 and a coefficient a₂₂ for the position P₂ ofthe exit pupil 133. Similarly, a coefficient b₀ is calculated from b₂₁and b₂₂ referring to Table 3, a coefficient c₀ is calculated from c₂₁and c₂₂ referring to Table 4, and a coefficient d₀ is calculated fromd₂₁ and d₂₂ referring to Table 5. Based above, the color shadingcorrection functions, i.e., R-gain-function(r) and B-gain-function(r),are calculated according to the following formulae (13) and (14),respectively.R-gain-function(r)=a ₀ r+b ₀  (13)B-gain-function(r)=c ₀ r+d ₀  (14)

In a step S8, the following decision is made for each of the pixels thatform correction target images. First, a decision is made whether or notthe image height (distance from the optical axis) r of a target pixel islarger than a threshold image height rmin. If the result of the decisionis YES, that is, if it is decided that the image height r of the pixelis larger than the threshold rmin, the process is advanced to a step S9.If the result of the decision is NO, that is, if the image height r ofthe pixel is equal to or smaller than the threshold rmin, the process isadvanced to a step S10 in which no color shading correction is performedon the each pixel of which the result of the decision is NO.

In the step S9, respective pixel values, i.e., R value and B value aremultiplied by the R-gain-function(r) and the B-gain-function (r),respectively, calculated in the step S7 to perform color shadingcorrection, and the process is advanced to a step S11.

In the step S11, a decision is made as to whether or not color shadingcorrection has been performed for all the pixels that require the colorshading correction. If the result of the decision is NO, that is, if itis decided that there is a pixel on which no color shading correctionhas been made, the process is returned to the step S8. If the result ofthe decision is YES, that is, if it is decided that color shadingcorrection has been performed for all the pixels that require the colorshading correction, the process is advanced to a step S12. In the stepS12, the image on which the color shading correction has been performedin the step S9 or the image on which the color shading correction hasnot been performed in the step S4 is displayed on the crystal monitor128 and a series of processing is ended.

As described in the above, the camera according to the first embodimentprovides the following advantages.

(1) It is configured such that color shading correction is performed onan image formed by a plurality of image signals having color components(R, B, and G in the embodiment). In this case, the calculation circuit101 makes a decision as to whether or not color shading correction is tobe performed for each pixel signal based on optical photographicconditions upon photography of a correction target image (lens type,exit pupil position, and aperture value in the embodiment). Thecalculation circuit 101 performs color shading correction on a pixelsignal of which it has been decided to perform correction for each ofthe R value and the B value, i.e., for each color component. As aresult, correction processing is performed on those pixel signals onwhich color shading has a significant impact, so that a load on theprocessing can be decreased to enable high speed processing as comparedwith the case in which the processing is performed for all the pixelsignals.

(2) The numerical aperture of the microlens 131 and the pitch of themicrolens 131 are unique for each of the image sensors 121. The colorshading occurs due to the incidence angle of light to each pixel of theimage sensor 121. Accordingly, in the present embodiment, it isconfigured such that correction data for the color shading correctionare calculated based on various factors that determine the incidenceangle to each pixel. As a result, the color shading that occurs in thephotographic image can be appropriately corrected even when thephotographic flux is incident to each pixel at a different angledepending on the optical photographic conditions.

(3) In the present embodiment, it is configured such that theR-gain-function(r) and the B-gain-function(r) are obtained based on theposition of the exit pupil 133 and the aperture value that may varydepending on the optical photographic conditions using data forcorrection prepared in advance based on the position of the exit pupil133 and the aperture value for each lens. As a result, the color shadingthat occurs in the photographic image can be appropriately correctedeven when the lens type, the exit pupil position, and the aperture valueare different.

(4) The data for color shading correction are prepared as follows. Agrey chart having a uniform color distribution is provided in advanceand the grey chart is photographed by varying the camera-to-subjectdistance and the focal length for each lens. To calculate gains of B andR as position functions for respective pixels of the image sensor 121,coefficients “a” to “d” of the B- and R-gain-functions(r) are calculatedbased on the images obtained by photography of the chart, and the R andB gains, which are correction amounts, are determined for each pixel.Therefore, the color shading that occurs in the photographic image canbe appropriately corrected.

(5) It is configured such that when the position of the exit pupil 133is located at a position farther than the determined threshold, thecolor shading correction is not performed. This is because the fartherthe position of the exit pupil 133 is from the image sensor 121, thesmaller the incidence angle to the image sensor 121 is and the smallerthe influence by the color shading is. Therefore, correction is notperformed on the image which is less influenced by color shading, sothat the load on processing can be decreased and the correction can beperformed efficiently.

(6) It is configured such that the color shading correction is performedonly on those pixel signals that are output from pixels in a larger areathan the predetermined image height set based on the opticalphotographic conditions. Even when the position of the exit pupil 133 isthe same and the aperture diameter is the same, the farther the positionof pixel is from the axis, the larger the incidence angle is, so thatthe influence of color shading becomes non-negligible. Therefore, pixelsof which image correction should be performed are discriminated frompixels of which image correction does not have to be performed dependingon the position of the pixel, so that the load on the processing can bedecreased and the correction can be performed efficiently at high speed.

(7) Dependence of the spectral transmittances of the infrared cut filter132 a and the ultraviolet cut filter 132 b on incidence angle alsoconstitutes a factor for color shading. As mentioned above, theincidence angle depends on the position of the exit pupil 133, theaperture value, and the position of the pixel, so that correction dataobtained from images obtained by photography of a grey chart or the likehaving a uniform color distribution in advance contains variation ofspectral transmittance depending on incidence angles to various filters132. Therefore, the influences of color shading caused by thecharacteristics of dependence of the spectral transmittances of thevarious filters 132 on the incidence angle can be appropriatelycorrected.

Second Embodiment

In the first embodiment, it is configured such that the data for colorshading correction prepared in advance, that is, Tables 1 to 5 arestored in the ROM in the camera or in the lens and the correction dataare used to determine the R-gain-function(r) and the B-gain-function(r).Then, the R signal and the B signal are multiplied with the gain of eachpixel calculated depending on the image height r.

In contrast thereto, in the second embodiment, a chart having an alreadyknown color distribution is photographed, prior to photography of asubject, under the optical photographic conditions of the same lens, thesame focal length, the same camera-to-subject distance, and the sameaperture diameter as those upon photography of the subject. Based on theobtained chart image, an R-gain-function(x,y) and a B-gain-function(x,y)are calculated at each image position. Then, each pixel value ofcorrection target image photographed under the same optical photographicconditions as those upon photography of the chart image is corrected bythe calculated R and B gains.

The personal computer with the image processing device according to thesecond embodiment of the present invention is described with referenceto FIGS. 11 and 12. As shown in the main part configuration diagram inFIG. 11, a personal computer 20 includes an external input circuit 201,a memory 202, a memory unit 203, a keyboard 204, a control circuit 205,and a monitor 206. Image data and optical photographic conditions areinput to the external input circuit 201 through the electronic camera 1connected thereto. The memory 202 includes a program storage area 202Athat stores a program for executing color shading correction and a datastorage area 202B that stores image data during image processing and theoptical photographic condition data. The memory unit 203 can store imagedata and optical photographic data permanently.

The keyboard 204 is an operation member through which execution ofvarious settings and various processings of the personal computer 20 isinstructed. The control circuit 205 includes a CPU or the like andcontrols respective units that constitute the personal computer 20. Whenan operation signal output from the keyboard 202 is input to the controlcircuit 205, the control circuit 205 reads in the program stored in thememory unit 203 in advance and stores a program in the program storagearea 202A in the memory 202. Further, the control circuit 205 executescolor shading correction processing of a chart image read in from thememory unit 203 or the external input circuit 201 based on the opticalphotographic condition data separately read in from the memory unit 203or the external input circuit 201 according to the program. Note thatthe optical photographic conditions contain the position of exit pupiland the aperture value upon photography of a chart image. The image thathas been subjected to correction processing is displayed on the monitor206.

The method for calculating the R-gain-function(x,y) and theB-gain-function(x,y) by the control circuit 205 will be described asfollows. Note that the distribution of respective color components inthe chart image is expressed by color distribution data such asR_CHART(x,y), G_CHART(x,y), and B_CHART(x,y), which are stored in thememory unit 203 or the like in the personal computer 20 in advance. Thecolor distribution data are correlated to the chart image and read in bythe control circuit 205 as necessary.

The control circuit 205 calculates ratios of R, G, and B, i.e.,R/G(x,y), and B/G(x,y) in respective pixel positions (x,y) based on theimage signals of the read in chart image. Further, the control circuit205 calculates ratios of R, G, and B, i.e., R_CHART(x,y)/G_CHART(x,y)and R_CHART(x,y)/B_CHART(x,y) that are expected in respective pixelpositions (x,y) based on the color distribution data.

Based on the ratios of R, G, and B in the respective pixel positions(x,y) calculated as mentioned above, the control circuit 205 calculatesthe R-gain(x,y) and the B-gain(x,y), in the respective pixel positions,i.e., color shading correction data according to formulae (15) and (16)shown below.R-gain(x,y)=(R_CHART(x,y)/G_CHART(x,y))/(R/G(x,y))  (15)B-gain(x,y)=(B_CHART(x,y)/G_CHART(x,y))/(B/G(x,y))  (16)

On the other hand, when the chart is achromatic such as grey or whiteand the color distribution is uniform, R_CHART(x,y), G_CHART(x,y), andB_CHART(x,y) are constant in all the pixel positions. That is, ratios ofR, G, and B, i.e., R/G(x0,y0) and B/G(x0,y0) in the center position(x0,y0) of the chart image can be expressed by the following formulae(17) and (18), respectively.R/G(x0,y0)=R_CHART(x,y)/G_CHART(x,y)  (17)B/G(x0,y0)=R_CHART(x,y)/B_CHART(x,y)  (18)

Accordingly, when the color distribution of the chart is uniform, thecontrol circuit 205 uses formulae (19) and (20) below obtained bysubstituting the formulae (17) and (18) into the formulae (15) and (16),respectively, to calculate color shading correction data.R-gain(x,y)=(R/G(x0,y0))/(R/G(x,y))  (19)B-gain(x,y)=(B/G(x0,y0))/(B/G(x,y))  (20)

When a chart image having a uniform color distribution is used as in theformulae (19) and (20) mentioned above, the color shading correctiondata are expressed as ratios of R, G, and B in the respective pixelpositions to R, G, and B in the image center position (x0,y0) of thechart image. Therefore, it is unnecessary to store in advance the colordistribution data R_CHART(x,y), G_CHART(x,y), and B_CHART (x,y) of thechart used in photographing the chart image or read them in from outsidefor correction. This can decrease the load incurred in the correctionprocessing.

The chart used for photographing a chart image may be either a chartthat is achromatic such as grey or white having a uniform colordistribution or a gradated chart having a nonuniform color distributionso far as the color distribution of the chart is known in advance. Whichof the charts is to be used is selected by a user. Therefore, when thechart having a nonuniform color distribution is used, the user performsan operation to instruct read in of color distribution data through thekeyboard 204. Alternatively, it may be configured such that the monitor206 displays an inquiry as to which one of the charts the user uses andwhen the user selects the chart having a nonuniform color distribution,the color distribution data are read in.

When the R-gain(x,y) and the B-gain(x,y) are calculated using theformulae (15) and (16), respectively, or the formulae (19) and (20),respectively, the control circuit 205 calculates threshold rmin of theread in correction target image using the formula (12) similarly to thecalculation circuit 101 in the first embodiment. The control circuit 205multiplies the R value and the B value in the pixel positions in whichthey are larger than the threshold rmin by the calculated R-gain(x,y)and B-gain (x,y), respectively, to perform color shading correction.

Referring to the flowchart shown in FIG. 12, the color shadingcorrection processing by the control circuit 205 is described. Note thatthe flowchart shown in FIG. 12 illustrates the processing procedure bythe program executed in the control circuit 205. The program is storedin the memory 202 and started up when an operation signal that instructsexecution of correction processing is input through the keyboard 204.

In a step S101, chart image data corresponding to a chart image obtainedby photography of a subject having a known color distribution underuniform illumination using a photographic screen in whole and opticalphotographic condition data corresponding to the chart image data areread in, and the process is advanced to a step S102. In the step S102,it is decided whether or not the color distribution of the photographedchart image is uniform. Since the chart has a uniform color distributionwhen the instruction to read in the color distribution data are notinput, the result of the decision in the step S102 is YES, and theprocess is advanced to a step S103. In the step S103, the color shadingcorrection data are calculated using the formulae (19) and (20)mentioned above, and the process is advanced to a step S106.

When the instruction to read in the color distribution data are input,the result of the decision in the step S102 is NO since the colordistribution of the chart is nonuniform, and the process is advanced toa step S104. In the step S104, the color distribution data are read in,and the process is advanced to a step S105. In the step S105, the colorshading correction data are calculated using the formulae (15) and (16)mentioned above and the process is advanced to a step S106.

In the step S106, the correction target image is read in, and theprocess is advanced to a step S107. In the step S107, similarly to thestep S6 in FIG. 10, the minimum image height rmin, which is a threshold,is calculated, and the process is advance to a step S108. In the stepS108, similarly to the step S8, it is decided whether or not the imageheight r of the correction target image is larger than the thresholdrmin. If the image height r is larger than the threshold rmin, theresult of the decision in the step S108 is YES, and the process isadvanced to a step S109. In the step S109, the R and B values in therespective pixel positions of the correction target image are multipliedby the R-gain(x,y) and the B-gain(x,y) calculated in the step S103 orthe step S105 to perform color shading correction, and the process isadvanced to a step S111.

On the other hand, if the image height r of the correction target imageis equal to or smaller than the threshold rmin, the result of thedecision in the step S108 is NO, and the process is advanced to a stepS110. Each processing in the step S110 (without color shadingcorrection) to the step S112 (image display) that is similar to eachprocessing in the step S10 (without color shading correction) to thestep S12 (image display) is performed and a series of processings iscompleted.

As described above, the personal computer having mounted thereon theimage processing device according to the second embodiment exhibits thefollowing advantages in addition to the effect (6) above obtained in thefirst embodiment.

(1) It is configured such that the control circuit 205 calculates theR-gain(x,y) and the B-gain(x,y) as color shading correction data fromthe first image data corresponding to the first image (chart image)obtained by photography of a chart having a known color distribution inadvance. The control circuit 205 performs color shading correction foreach color component of the pixel signal of the second image (correctiontarget image) based on the calculated R and B gains. Therefore, thecolor shading that occurs in the photographic image can be appropriatelycorrected without retaining or referring to the data for correction inTables 2 to 5 unlike the electronic camera 1 of the first embodiment.

(2) When the chart having a uniform color distribution is used as thefirst image data, the steps S102, S104, and S105 in FIG. 12 can beomitted, so that the color shading can be appropriately corrected withno need to retain the color distribution of the chart or to read it infreshly. Note that taking into consideration precision of the correctiondata, a grey, white or the like achromatic chart having equal amounts ofcolor components of R, G, and B is suitable as the chart having auniform color distribution. However, the chart is not limited toachromatic one so far as the color distribution is uniform. Coloredcharts may be usable for obtaining correction data.

The image processing device that performs color shading correctionprocessing in the above-mentioned second embodiment may be mounted onthe electronic camera 1. FIG. 13 shows constituent elements of suchelectronic camera 1. The constituent elements that are the same as thoseof the electronic camera in the first embodiment are assigned the samereference numerals. Note that the electronic camera 1 is configured tohave a shading correction photographic mode as one of photographic modesto enable setting of the shading correction photographic mode, forexample, by operating the mode dial 4. When the shading correctionphotographic mode is set, the electronic camera 1 creates R and B gainsfrom pixel signals obtained by photography of a chart having a uniformcolor distribution and performs color shading correction processing to acorrection target image photographed under the same optical photographiccondition as that when photography is performed to take an image of thechart.

As shown in FIG. 13, the image processing circuit 123 includes a gainpreparing section 123A and a correction section 123B. The imageprocessing circuit 123 includes, for example, ASIC. The gain preparingsection 123A and the correction section 123B are constituted bydedicated logic circuits (hardware). Note that the image processingcircuit 123 may be constituted by a DSP (Digital Signal Processor) orthe like. Once the shading correction photographic mode is set, the gainpreparing section 123A generates the R-gain(x,y) and the B-gain(x,y)based on the pixel signals that form the input chart image dataaccording to the formulae (19) and (20), respectively, mentioned above.The generated R-gain(x,y) and the B-gain(x,y) are set in the correctionsection 123B as color shading correction data.

Subsequently, when the correction target image is photographed under thesame optical photographic conditions as the chart image, the pixelsignals that form the correction target image are input to thecorrection section 123B. The correction section 123B multiplies R valueand B value of the pixel signals corresponding to the correction targetimage by the R-gain(x,y) and the B-gain(x,y), respectively, using thecolor shading correction data generated by the gain preparing section123A to perform color shading correction. Note that the color shadingcorrection is performed for all the pixel signals that constitute thecorrection target image. The image data subjected to the color shadingcorrection processing are output to the calculation circuit 101 andsubjected to various types of processing such as compression,displaying, recording and so on.

Explanation is made on the action of color shading correction processingin the electronic camera 1 thus far described referring to the flowchartshown in FIG. 14. Each processing shown in the flowchart in FIG. 14 isexecuted by the image processing circuit 123 when the shading correctionphotographic mode is set. In a step S21, image signals forming a chartimage obtained by photography of a chart are input, and the process isadvanced to a step S22. In the step S22, the input pixel signals and theformulae (19) and (20) are used to generate the color shading correctiondata R-gain (x,y) and B-gain(x,y), respectively, and the process isadvanced to a step S23.

In the step S23, pixel signals that form the correction target imagetaken under the same optical photographic condition as the chart imageare input, and the process is advanced to a step S 24. In the step S24,the R-gain(x,y) and the B-gain(x,y) generated in the step S22 are usedto perform color shading correction to all the pixel signals that formthe input correction target image, and the process is advanced to a stepS25. In the step S25, the image data subjected to the color shadingcorrection are output to the calculation circuit 101 to complete aseries of processings.

In this case, if a standard chart for preparing color shading correctiondata are performed whenever photography of the subject is performed andreal time color shading correction processing is performed wheneverphotography is performed, photographic performance is influenced bycalculation time. Accordingly, it may be configured such that the twoimages obtained, i.e., a principal image and a reference image, arestored in pairs and they are subjected to color shading correctionprocessing later by a personal computer or the like.

By storing R-gain(x,y) and B-gain(x,y) under various opticalphotographic conditions as historical information, it becomesunnecessary to load the reference image when photography is performedunder the optical photographic conditions contained in the historicalinformation. Further, tables equivalent to Tables 2 to 5 shown in FIGS.9A to 9D can be prepared. In this case, color shading correction can beperformed in the same manner as that in the first embodiment even whenphotography is performed under optical photographic conditions that arenot contained in the historical information.

Third Embodiment

The camera with the image processing device according to a thirdembodiment of the present invention is described focusing on differencesfrom the first and the second embodiments. The electronic camera 1 inthe third embodiment is assumed to include the same constituent elementsas those of the electronic camera in the first embodiment shown in FIGS.1 and 2. The electronic camera 1 is configured to calculate R and B gainfunctions referring to data for correction (Tables 2 to 5) according tothe formulae (1) and (2), respectively, in the same manner as in that inthe first embodiment and correct the obtained R and B gain functionswith R and B gains, respectively, which are color shading correctiondata calculated based on a chart image according to the formulae (15)and (16), respectively, or the formulae (19) and (20), respectively, inthe same manner as that in the second embodiment.

Then, color shading of a correction target image is corrected with thecorrected R and B gain functions. That is, the R and B gain functions,which are calculated from data for correction that are averaged out permodel of the electronic camera 1, are corrected to calculate colorshading correction data that enables corrections of color shadingdepending on an individual difference of each electronic camera 1.Further, correction becomes possible even when color shading that is notin rotational symmetry occurs on the photographic screen.

First, explanation is made on calculation of correction functionsR_calib(x,y) and B_calib(x,y) for correcting the R-gain-function(r) andthe B-gain-function(r), respectively, which are color shading correctiondata with reference to FIG. 2. The electronic camera 1 has a correctionfunction calculation mode in addition to a photographic mode and areproduction mode, and the correction function calculation mode can beset, for example, by operation of the mode dial 4. When a signalindicating that the correction function calculation mode is set is inputto the calculation circuit 101 from the mode dial 4, the calculationcircuit 101 reads in the chart image obtained by photography of thechart having a known color distribution and the optical photographiccondition data in the same manner as in the second embodiment. Then, thecalculation circuit 101 calculates R-gain1 (x,y) and B-gain1 (x,y) ineach pixel positions (x,y) using the formulae (15) and (16),respectively, or the formulae (19) and (20), respectively.

The electronic camera 1 according to the third embodiment, like theelectronic camera 1 according to the first embodiment, stores in advanceTables 2 to 5 depending on the model of the electronic camera 1 in apredetermined storage area in the calculation circuit 101. Thecalculation circuit 101 calculates coefficients “a” to “d” based on theaperture value and the exit pupil position upon photography of a chartimage recorded in the read in optical photographic condition data withreference to Tables 2 to 5. The calculation circuit 101 calculatesR-gain2(x,y) and B-gain2(x,y) in each pixel position (x,y) of the chartimage using the formulae (1) and (2), respectively. Note that when theposition P₀ of the exit pupil 133 is remoter from the photographicscreen, the incidence angle is smaller in the same manner as that in thefirst embodiment mentioned above, so that the influence of color shadingcan be ignored. Accordingly, the calculation circuit 101 sets theR-gain2(x,y) and the B-gain2(x,y) as indicated by formula (21) below.R-gain2(x,y)=the B-gain2(x,y)=1  (21)

Based on the R-gain1(x,y), B-gain1(x,y), R-gain2 (x,y), and B-gain2(x,y) calculated as mentioned above, the calculation circuit 101calculates correction functions R_calib(x,y) and B_calib(x,y) accordingto the formulae (22) and (23), respectively, below.R_calib(x,y)=R-gain1(x,y)/R-gain2(x,y)  (22)B_calib(x,y)=B-gain1(x,y)/B-gain2(x,y)  (23)

The calculated R_calib(x,y) and B_calib(x,y) are recorded by thecalculation circuit 101 in the recording medium 126. The calculationcircuit 101 uses the correction functions R_calib(x,y) and B_calib(x,y)to correct the R-gain-function and the B-gain-function, respectively,that are calculated form the correction target image in the same manneras that in the first embodiment. Since the correction functions areapplicable when the optical photographic conditions of the chart imageare different from the optical photographic conditions of the correctiontarget image, it is only needed to perform calculation processing atleast once. Note that a plurality of correction functions calculatedusing a plurality of chart images obtained under different opticalphotographic conditions may be stored. In this case, the calculationcircuit 101 may be configured to correct the R-gain-function and theB-gain-function of the correction target image by using the correctionfunction calculated based on the chart image photographed under opticalphotographic conditions close to the optical photographic conditions forthe correction target image.

Now, the color shading correction processing by the calculation circuit101 is described. The calculation circuit 101 reads in the image data ofthe correction target image obtained by photography. Further, thecalculation circuit 101 receives the exit pupil position and theaperture value at the time when the correction target image is takenfrom the lens side and reads them in, in the same manner as that in thefirst embodiment. The calculation circuit 101 calculates thecoefficients “a” to “d” based on the aperture value and the exit pupilposition transmitted from the lens side referring to Tables 2 to 5stored in advance to calculate the R-gain(r) and the B-gain(r) of thecorrection target image.

The calculation circuit 101 reads out the correction functionsR_calib(x,y) and B_calib(x,y) stored in the recording medium 126. Asshown in the formulae (24) and (25) below, the calculation circuit 101multiplies the R-gain(r) and the B-gain(r) by the correction functionsR_calib(x,y) and B_calib(x,y), respectively, to calculate color shadingcorrection data, R-gain-h(x,y) and B-gain-h(x,y), respectively.R-gain-h(x,y)=R-gain(r)×R_calib(x,y)  (24)B-gain-h(x,y)=B-gain(r)×B_calib(x,y)  (25)

The calculation circuit 101 performs color shading correction bymultiplying the R value and the B value in each pixel position of thecorrection target image by the R-gain-h(x,y) and the B-gain-h (x,y),respectively, which are the calculated color shading correction data.The color shading correction processing is performed for all the pixelsignals that form the target correction image.

The calculation processing for calculating the correction functions inthe electronic camera 1 according to the third embodiment mentionedabove is described referring to the flowchart shown in FIG. 15. Notethat the flowchart shown in FIG. 15 illustrates a procedure by a programthat is executed in the calculation circuit 101. The program is recordedin the memory (not shown) and is started up when a signal indicatingthat a correction function calculation mode is set is input thereinthrough the mode dial 4.

Each processing in a step S201 (reading in of chart image data andoptical photographic condition data) to a step S205 (calculation ofR-gain1(x,y) and B-gain1(x,y)) is performed similarly to each processingin the step S101 (reading in of the chart image data and the opticalphotographic condition data) to the step S105 (calculation of the colorshading correction data). In a step S206, the exit pupil positioncontained in the optical photographic condition data read in during thestep S201 are read in, and the process is advanced to a step S207.

In the step S207, it is decided whether or not the exit pupil positionis equal to or larger than the predetermined threshold Pmax similarly tothe step S3 in FIG. 10. If the exit pupil position is equal to or largerthan the threshold Pmax, the result of the decision in the step S207 isYES, and the process is advanced to a step S208. In the step S208, theR-gain2 (x,y) and the B-gain2 (x,y) are each set to 1 as shown in theformula (21), and the process is advanced to a step S211. If the exitpupil position is below the threshold Pmax, the result of the decisionin the step 207 is NO, and the process is advanced to a step S209. Inthe step S209, the aperture value upon photography of the chart imagecontained in the optical photographic condition data are read out, andthe process is advanced to a step S210.

In the step S210, the R-gain2(x,y) and the B-gain2 (x,y) are calculatedaccording to the formulae (1) and (2), respectively, and the process isadvanced to the step S211. In the step S211, correction functionsR_calib(x,y) and B_calib(x,y) are calculated based on the R-gain1(x,y)and the B-gain1(x,y), respectively, calculated in the step S203 or thestep S205 and the R-gain2(x,y) and the B-gain2(x,y) calculated in thestep S208 or the S210 according to the formulae (22) and (23),respectively, and the process is advanced to a step S212. In the step212, the correction functions R_calib(x,y) and B_calib(x,y) calculatedin the step S211 are recorded in the recording medium 126, and a seriesof processing is ended.

Then, the color shading correction processing in the electronic camera 1according to the third embodiment is described referring to theflowchart shown in FIG. 16. Each processing in FIG. 16 is performed byexecuting the program in the calculation circuit 101. The program isstored in a memory (not shown) and started up when an image that is acorrection target image is photographed.

Each processing in a step S301 (reading in of a correction target image)and a step S302 (reading in of optical photographic condition date) isperformed similarly to each processing in the step S1 (reading in of thecorrection target image) and the step S2 (reading in of the opticalphotographic condition data) in FIG. 10. In a step S303, the aperturevalue at the time upon photography of the correction target image isread in, and the process is advanced to a step S304. In the step S304,the R-gain(r) and the B-gain(r) of the correction target image arecalculated based on the exit pupil position and the aperture value readin according to the formulae (1) and (2) respectively, and the processis advanced to a step S305.

In the step S305, the correction functions R_calib(x,y) and B_calib(x,y)recorded in the recording medium 126 are read out, and the process isadvanced to a step S 306. In the step S306, color shading correctiondata, R-gain-h(x,y) and B-gain-h(x,y), are calculated according to theformulae (24) and (25), respectively, based on the R-gain(r) and theB-gain(r), respectively, calculated in the step S304 and on thecorrection functions R_calib(x,y) and B_calib(x,y), respectively, readin during the step S305, and the process is advanced to a step S307.

In the step S307, the R value and the B value at each pixel position ofthe correction target image are multiplied by the color shadingcorrection data, R-gain-h(x,y) and B-gain-h(x,y), respectively,calculated in the step S306 to perform color shading correction, and theprocess is advanced to a step S308. Each processing in the step S308(decision as to whether or not processing is completed for all thepixels) and the step S309 (image display) is performed similarly to eachprocessing in the step S11 (decision as to whether or not processing iscompleted for all the pixels) and the step S12 (image display) in FIG.10, and a series of processing is ended.

The camera 1 according to the third embodiment as mentioned aboveprovides the following advantages.

(1) The calculation circuit 101 is configured to update the R-gain2(x,y)and the B-gain2(x,y) calculated referring to Tables 2 to 5, which aredata for correction stored in advance using the R-gain1(x,y) and theB-gain1(x,y), respectively, calculated using the chart image data tocalculate correction functions. Also, the calculation circuit 101 isconfigured to correct color shading for each color component of theimage signals of the correction target image using the opticalphotographic conditions of the correction target image and thecalculated correction functions. Therefore, correction is performedusing optimal data for individual models of the electronic camera 1 evenwhen there are observed individual differences in color shadingcharacteristics despite of the same model being used or even whenprecision of correction data retained in advance is low, so that theprecision of correction can be improved.

(2) The calculation circuit 101 is configured to perform color shadingcorrection for all the pixel signals of the correction target image.Therefore, even when there is color shading that does not depend on theimage height, such as one due to unevenness in spectral sensitivity ofthe image sensor 121, correction is exactly performed, so that an imagehaving a high image quality overall the screen can be obtained.

The present invention can be practiced in the following aspects.

(1) Although the program for correcting color shading is executed by theelectronic camera 1 that retains the data for correction therein in thefirst embodiment, the personal computer 20 as shown in FIG. 11 may beused as in the second embodiment. That is, the color shading correctionprocessing of the image data of the image taken may be executed usingthe personal computer 20 having stored therein the data for correction.In this case, the correction is performed as follows.

In the memory section 203 in the personal computer 20 shown in FIG. 11,there are stored in advance Tables 1 to 5, data for color shadingcorrection, for every type of the electronic camera 1. The program bywhich the control circuit 205 executes the processing shown in theflowchart of FIG. 10 is stored in the program storage area 202A of thememory 202. In the correction target image, there are recorded dataindicating optical photographic conditions including lens type, focallength, camera-to-subject distance, and aperture value as well as cameramodel in relation to image data. The correction target image loaded inthe personal computer 20 through the external input circuit 201 arestored by the control circuit 205 in the data storage area 202B of thememory 202. The control circuit 205 performs correction data decisionprocessing by selecting Table 1 based on the lens type, selecting atable to be used from Tables 2 to 5 based on the camera model, andcalculating coefficients “a” to “d” referring to Tables 1 to 5 to obtainR gain and B gain. The control circuit 205 performs correction executiondecision processing to decide whether to execute correction for eachpixel signal that forms the correction target image based on the imageheight calculated from the exit pupil position and the aperture value.Further, the control circuit 205 performs color shading correctionprocessing on the pixel signal for which execution of correction isdecided in the correction execution decision processing. In this manner,the color shading correction can be executed by the personal computer 20without fail even for an image taken by using a different camera orlens.

(2) Also in the third embodiment, the processing performed by executionof the program in the electronic camera 1 may be executed by thepersonal computer 20 similarly to (1) above. In this case, in theprogram storage area 202A of the memory 202 in the personal computer 20,there is stored in advance the program executing the procedures shown inthe flowcharts in FIGS. 15 and 16. In the memory section 203, there arestored in advance the chart image, the optical photographic conditiondata, and the data for correction, Tables 2 to 5. Then, the controlcircuit 205 performs correction data acquisition processing to read inthe chart image and the optical photographic condition data in the datastorage area 202B of the memory 202. The control circuit 205 calculatescorrection functions using the chart image read in in the memory 202according to the above-mentioned program and performs modificationprocessing using the calculated correction functions to modify the colorshading correction data calculated based on the data for correctionstored in advance. Finally, the control circuit 205 executes colorshading correction processing using the modified correction data on thecorrection target image acquired through the external input circuit 201.In this method, images taken by electronic cameras to which colorshading gives different influences can be corrected always at the samequality.

(3) When applied to the personal computer 20 or the like, the programrelating to the above-mentioned control can be furnished through arecording medium such as a CD-ROM or data signals over the Internet.FIG. 17 illustrates the manner of such. The personal computer 20 isfurnished with the program through the CD-ROM 22. The personal computer20 has a capability of being connected to the communication line 23. Thecomputer 24 is a server computer that provides the program and storesthe program in a recording medium such as the hard disk 22. Thecommunication line 23 includes the Internet, communication line such aspersonal computer communication, and a dedicated communication line. Thecomputer 24 reads out the program using the hard disk 22 and transmitsthe program to the personal computer 20 through the communication line23. That is, the computer 24 transmits the program as data signals on acarrier wave through the communication line 23. In this manner, theprogram can be provided as a computer-readable computer program productin various forms including a recording medium and a carrier wave.

(4) In the electronic cameras 1 according to the first and the thirdembodiment, the program of the present invention executed in theelectronic camera 1 is loaded on a ROM usually upon production. However,the ROM loading the program may be a rewritable ROM, which can beconnected to the personal computer 20 in a configuration similar to thatshown in FIG. 17 in order to provide an improved program from arecording medium such as a CD-ROM through the personal computer 20.Further, the improved program may be provided through the Internet orthe like in the same manner as mentioned above.

(5) In the electronic camera 1 according to the first embodiment, thecalculation circuit 101 may execute the color shading correctionprocessing and the white balance processing in combination. In thiscase, the calculation circuit 101 calculates white balance correctionvalues R_wb and B_wb by which the R value and the B value, respectively,of the pixel signals are to be multiplied by known white balanceprocessing. The calculation circuit 101 performs white balanceprocessing in compensation for not performing color shading correctionin the step S4 shown in the flowchart in FIG. 10. Further, thecalculation circuit 101 performs color shading correction processing inthe step S9 by multiplying the R value and the B value in each pixelposition by respective correction values obtained by multiplying theR-gain(r) and the B-gain(r) calculated in the step S7 by white balancecorrection values R_wb and B_wb, respectively.

(6) In the second embodiment, the control circuit 205 may be configurednot to perform decision as to whether or not correction is to beexecuted in the correction target image but to perform color shadingcorrection on the pixel signals in all the pixel positions. That is, theprocessing in the step S108 and the step S110 in the flowchart shown inFIG. 12 may be omitted. When a component that does not depend on theincidence angle of the image sensor 121, such as spectral sensitivitydistribution unevenness of the image sensor 121, has a significantimpact, color shading occurs also in a component other than rotationsymmetry component. This component cannot be expressed by a function ofthe image height r and appears two-dimensional in the screen. In such acase, it is desirable that color shading correction is performed on allthe pixels as mentioned above.

(7) In the third embodiment, when the calculated correction functionsR_calib(x,y) and B_calib(x,y) can be expressed as functions of the imageheight r, correction of the color shading correction data will beprincipally correction of the coefficients obtained from Tables 2 to 5.Therefore, the color shading correction can be omitted for the case inwhich the exit pupil distance is long and the case in which the imageheight is small similarly to the first embodiment in order to speed upthe processing.

(8) In the third embodiment, a chart image or images obtained byphotography of a uniform achromatic chart under one or more opticalphotographic conditions may be used as preliminarily retained data forcorrection in place of Tables 2 to 5 described in the first embodiment.This enables improvement in precision when the preliminarily retainedchart image or images are insufficient for providing a desiredcorrection precision.

(9) Each processing in the third embodiment may be performed by theimage processing circuit of the electronic camera shown in FIG. 13.

(10) While the present invention has been explained as applicable to anelectronic camera with an interchangeable lens, the present inventionmay also be applied to an electronic camera with a lens. In the case ofthe electronic camera with a lens, it is only necessary to store a setof Tables 1 to 5 mentioned above in the camera to allow the calculationcircuit 101 in order to calculate exit pupil position using the opticalphotographic conditions and Table 1.

(11) Although explanation has been focused on the image sensor with RGBfilters, the present invention is also applicable to image sensors withvarious color separation filters including complementary color filters.

While various embodiments and modifications have been explained above,the present invention is not limited thereto and other modes conceivablewithin the technical concepts of the present invention should beincluded in the present invention.

1. An image processing device that performs a color shading correctionfor a plurality of pixel signals output from an imaging sensor with acolor separator, the imaging sensor capturing a subject via an opticalsystem and outputting the pixel signals forming an image and beinginfluenced by the color shading depending on optical characteristics ofthe optical system and optical characteristics of the color separator,the image processing device comprising: a data obtaining unit thatobtains a first image having a color distribution that is known inadvance; a decision making unit that decides whether or not the colordistribution of the first image obtained by the data obtaining unit isuniform; a correction data obtaining unit that obtains correction databy using the first image if the color distribution is decided to beuniform, and that obtains the plurality of correction data by using thefirst image and color distribution data regarding the first image if thecolor distribution is decided not to be uniform; a correction executiondecision unit that decides that the color shading correction is to beexecuted for the pixel signals in a peripheral area of a second imageand decides that the color shading correction is not to be executed forthe pixel signals in a respectively central area of the second image;and a correction unit that corrects each color component of theplurality of pixel signals of the second image for which the colorshading correction is decided to be executed by the color shadingcorrection execution decision unit, by using the correction dataobtained by the correction data obtaining unit.
 2. An image processingdevice according to claim 1, wherein the correction data obtaining unitobtains the correction data for each color component suitable for apredetermined optical photographic condition for each of the pluralityof pixel signals using the first image obtained by capturing a subjecthaving a known color distribution under the predetermined opticalphotographic condition, and the correction unit corrects for each colorcomponent of a plurality of image signals that form the second imageobtained under the same optical photographic condition as thepredetermined optical photographic condition based on the correctiondata.
 3. An image processing device according to claim 1, furthercomprising: an updating unit that updates data relating to correctionstored in advance based on the first image to generate the correctiondata, wherein the correction unit corrects for each color component aplurality of pixel signals that form the second image based on theoptical photographic condition when the second image is obtained and theupdated correction data.
 4. An image processing device according toclaim 1, wherein the subject having a known color distribution is achart having a uniform color distribution.
 5. An image processing deviceaccording to claim 1, wherein the color separator is configured tomicrolenses, color filters, an infrared ray cut filter and anultraviolet ray cut filter that are disposed on the image sensor.
 6. Animage processing device according to claim 1, wherein the opticalphotographic condition includes an exit pupil position of the opticalsystem and an aperture value of the optical system when the image isobtained.
 7. An image processing device according to claim 1, furthercomprising: a size changing unit that changes a size of the peripheralarea in which the color shading correction is to be executed, based uponan optical photographic condition when the second image was captured. 8.An image processing device according to claim 1, wherein: each set ofthe correction data is suitable for an optical photographic conditionfor each of the plurality of pixel signals and is different from eachother in each color component.
 9. An image processing method thatperforms a color shading correction for an image formed by a pluralityof pixel signals output from an imaging sensor with a color separator,the imaging sensor capturing a subject via an optical system andoutputting the pixel signals forming an image and being influenced bythe color shading depending on optical characteristics of the opticalsystem and optical characteristics of the color separator, the imageprocessing method comprising: obtaining a first image having a colordistribution that is known in advance; deciding whether or not the colordistribution of the obtained first image is uniform; obtaining the firstimage if the color distribution is decided to be uniform and obtainingthe plurality of correction data by using the first image and colordistribution data regarding the first image if the color distribution isdecided not to be uniform; deciding that the color shading correction isto be executed for the pixel signals in a peripheral area of a secondimage and is not to be executed for the pixel signals in a respectivelycentral area of the second image; and correcting for each colorcomponent of the plurality of pixel signals forming the second image forwhich the color shading correction is decided to be executed, by usingthe obtained correction data.
 10. An image processing method accordingto claim 9, wherein the correction data for each color componentsuitable for a predetermined optical photographic condition is obtainedfor each of the plurality of pixel signals using the first imageobtained by capturing a subject having a known color distribution underthe predetermined optical photographic condition, and a plurality ofimage signals that form the second image obtained under the same opticalphotographic condition as the predetermined optical photographiccondition are corrected for each color component based on the correctiondata.
 11. An image processing method according to claim 9, wherein datarelating to correction stored in advance is updated based on the firstimage to generate the correction data, and a plurality of pixel signalsthat form the second image is corrected for each color component basedon the optical photographic condition when the second image is obtainedand the updated correction data.
 12. An image processing methodaccording to claim 9, wherein the subject having a known colordistribution is a chart having a uniform color distribution.
 13. Animage processing method according to claim 9, further comprising:changing a size of the peripheral area in which the color shadingcorrection is to be executed, based upon an optical photographiccondition when the second image was captured.
 14. An image processingmethod according to claim 9, wherein: each set of the correction data issuitable for an optical photographic condition for each of the pluralityof pixel signals and is different from each other in each colorcomponent.
 15. A non-transitory computer readable medium on which isstored an image processing program adapted to cause a computer toexecute an image processing method that performs a color shadingcorrection for an image formed by a plurality of pixel signals outputfrom an imaging sensor with a color separator, the imaging sensorcapturing a subject via an optical system and outputting the pixelsignals forming an image and being influenced by the color shadingdepending on optical characteristics of the optical system and opticalcharacteristics of the color separator, the image processing methodcomprising: obtaining a first image having a color distribution that isknown in advance; deciding whether or not the color distribution of theobtained first image is uniform; Obtaining correction data by using thefirst image if the color distribution is decided to be uniform andobtaining the plurality of correction data by using the first image andcolor distribution data regarding the first image if the colordistribution is decided not to be uniform; Deciding that the colorshading correction is to be executed for the pixel signals in aperipheral area of a second image and is not to be executed for thepixel signals in a respectively central area of the second image; andCorrecting for each color component of the plurality of pixel signalsforming the second image for which the color shading correction isdecided to be executed, by using the obtained correction data.